Video fingerprinting using Latent Dirichlet Allocation and facial images

This paper investigates the possibility of extracting latent aspects of a video in order to develop a video fingerprinting framework. Semantic visual information about humans, more specifically face occurrences in video frames, along with a generative probabilistic model, namely the Latent Dirichlet Allocation (LDA), are utilized for this purpose. The latent variables, namely the video topics are modeled as a mixture of distributions of faces in each video. The method involves also Scale Invariant Features Transform (SIFT) based clustering of detected faces and adapts the bag-of-words concept into a bag-of-faces one, in order to ensure exchangeability between topics distributions. Experimental results provide evidence that the proposed method performs very efficiently for video fingerprinting. Preprint submitted to Elsevier July 15, 2011